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Abstract 

The consistency across scales of a recently developed mathematical thermodynamic structure, 
between a continuous stochastic nonlinear dynamical system (diffusion process with Langevin or 
Fokker-Planck equations) and its emergent discrete, inter-attractoral Markov jump process, is in- 
vestigated. We analyze how the system's thermodynamic state functions, e.g. free energy F, 
entropy S, entropy production e p , and free energy dissipation F, etc., are related when the contin- 
uous system is describe with a coarse-grained discrete variable. We show that the thermodynamics 
derived from the underlying detailed continuous dynamics is exact in the Helmholtz free-energy 
representation. That is, the system thermodynamic structure is the same as if one only takes a 
middle-road and starts with the "natural" discrete description, with the corresponding transition 
rates empirically determined. By "natural" , we mean in the thermodynamic limit of large systems 
in which there is an inherent separation of time scales between inter- and intra-attractoral dy- 
namics. This result generalizes a fundamental idea from chemistry and the theory of Kramers by 
including thermodynamics: while a mechanical description of a molecule is in terms of continuous 
bond lengths and angles, chemical reactions are phenomenologically described by the Law of Mass 
Action with rate constants, and a stochastic thermodynamics. 

PACS numbers: 05.70.Ln, 02.50.Ey, 82.20.Uv, 89.70.Cf 



I. INTRODUCTION 



Recently, a quite complete mathematical thermodynamic structure for general stochas- 
tic processes has been proposed, for both discrete Markov jump processes and continuous 
Langevin-Fokker-Planck systems p~H3]. The entropy production rate e p of a Markov dynam- 
ics can be mathematically decomposed into two non-negative terms: free energy dissipation 
rate —F, corresponding to Boltzmann's original theory on irreversibility of spontaneous 
change, and house-keeping heat Qhk, corresponding to Brussels school's notion of irreversibil- 
ity in nonequilibrium steady states (NESS) [3HB]. For almost all applications of stochastic 
dynamic theories in physics, chemistry and biology, there will be multiple time scales, and 
often with a significant separation. When a dynamical system is highly nonlinear, and its 
interaction network includes feedbacks, multistability with several attractors is often the rule 
rather than exception. On the other hand, the concept of "landscape" has become a highly 
popular metaphor as well as a useful analytical device [8] . When stochastic nonlinear dy- 
namical systems of populations of individuals become large, a time scale separation between 
inter- and intra-attractoral dynamics becomes almost guaranteed. In cellular biology, they 
have been called biochemical network and cellular evolution time scales respectively [9]. 

In chemistry, a separation of time scales has lead to a fundamental understanding of 
chemical reactions in terms of discrete states of molecules, in complementary to the full 
mechanical description of constitutive atoms in terms of bond lengths and bond angles. 
In fact, one of the most significant, novel chemical concepts is "transition state", which 
in terms of modern nonlinear dynamical systems is the saddle point on a separatrix that 
divides two attractors |10j . Recall also that in applications of Gibbs' formalism of statistical 
mechanics to chemical equilibrium, the conditional free energy plays a central role [HI [12] . 
One usually does not work with the pure mechanical energy of a system; rather, one works 
with a conditional free energy from a coarse-grained representation and develops a partition 
function thereafter. An essential notion in this approach is the consistency across scales. 
We shall expand on these ideas more precisely in the following section. 

In the present work we address the question of "whether the mathematical thermody- 
namic structure of a given continuous stochastic nonlinear dynamical system is consistent 
with the one associated with the emergent discrete Markov jump process." In other words: 
whether the formal mathematical relations between state functions and process variables 
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remain unchanged when the system is viewed at either a finer- or a coarse-grained scale. 

It is important to point out, at the onset, that the "state" of a stochastic dynamical system 
has always had two distinctly different meanings: (a) a state of a single, stochastically 
fluctuating, system; and (6) a state in terms of the distribution over an ensemble. In 
more precise mathematical terms, (a) are functions of a stochastic process, while (b) are 
functionals of the solution to a Fokker-Planck equation. The deep insight from the theory of 
probability is that these are two complementary, yet mathematically identical, descriptions 
of a same dynamical process. With this distinction in mind, entropy and free energies are 
state functionals of the second type, while energy is a state function of the first type. A 
state function of the first type naturally has fluctuations. On the other hand, most classical 
thermodynamic functions are the second type. 

Attempting to introduce entropy as a function of the first type, Qian [13] defined a 
trajectory based entropy T t = — \nf x (X t ) where X t is a diffusion process, and f x ( x ) is 
the stationary solution to the corresponding Fokker-Planck (Kolmogorov forward) equation. 
One immediately sees that entropy is really a population-based concept. For irreversible 
diffusion processes (i.e., without detailed balance), fx( x ) is non-local [13J. However, for 
reversible diffusion processes with detailed balance, since fx( x ) oc e~^ x ^ where <j>{x) is 
potential energy, fluctuating T t and fluctuating energy 4>(X t ) are the same. 

II. EQUILIBRIUM STATISTICAL THERMODYNAMIC CONSISTENCY 
ACROSS SCALES 

In equilibrium statistical mechanics, the concept of consistency — or invariance — has a 
fundamental importance in the study of realistic physical systems at an appropriate scale 
[TT| IT2] . In a continuous system, the conditional free energy is known as the potential of 
mean force [H]. The conditional free energy can do work just as the Newtonian mechanical 
energy; the concept of entropic force is well understood in physical chemistry [15] . 

For an investigator working on certain level of description, with discrete states (i = 
1, 2, • • • ) and conditional free energy (A4), the canonical partition function of the statistical 
thermodynamic system is [HI [321 EES] 




(1) 



i=l 
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Note that, since Ai is a conditional free energy, it can be decomposed into Ai = Ei — TSi, 
where Ei = d(A i /T) / d(l/T) and Si = —OAi/dT. In general, both E { and Si are themselves 
functions of the temperature. 

Now, for another investigator who works at a much more refined level, with a continuos 
variable x, each state i corresponds to a unique region of the phase space Ui, with f]ujj = 
for i 7^ j, and |Ji=i w « = ^ covering the entire phase-space region available to the system. 
Let V(x) (x G Q) be the potential of mean force at this level. Then, his/her canonical 
partition function is 

Z(T) = [ dxe- y W/ fcsT . (2) 
Jn 

We see that Z(T) and Z(T) are equal if the Ai in Eq. are such that 

Ai(T) = -k B T\n (^J rfxe- y(x)/fcflT ^ . (3) 

The equality Z(T) = Z{T) following Eqs. Q and Q is exact in equilibrium statistical me- 
chanics. Nonetheless, as we demonstrate in this work, its generalization to include dynamics 
requires a separation of time scales for the dynamics within each Ui and the dynamics be- 
tween w's (this is well understood in physical chemistry as "rapid equilibrium" averaging). 
Here, we choose the ou's according to the basins of attraction of the underlying nonlinear dy- 
namics. In this case, the separation of time scales for intra- and inter-attractoral dynamics 
is widely accepted. 

The mathematical origin of the consistency discussed above relies in fact upon the con- 
cepts of conditional probability, marginal probability, and the law of total probability] The 
free energy of a system, or a sub-system, is directly related to its probability. The same 
cannot be said for the entropy [TH [17] , which increses with more detailed descriptions and 
is also coordinate-system dependent for continuous variabels: 

/ -A t /k B T\ / p -A i /k B T\ 

^ = -^(-wrH-wr) 

r / -V(x)/fc B T\ / -V(x)/k B T\ 

£ - fe / n dx (^r) ln (^r)^ (r) - (4) 

The proof for the inequality can be found in any text on information theory |18j . Also see 
Appendix \A\ 

Note that since internal energy is the sum — k B T In Z(T) + TS(T) = E(T), one imme- 
diately has E(T) > E(T) across scales as well. This leads to a type of entropy-energy 
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compensation [131 El HH 120] • 



III. OPEN SYSTEM CONCEPTS AND DEFINITIONS 

A. Fokker-Planck equation, stationary distribution, and detailed balance 

Consider a system whose state is represented by variable x, and assume that x is a stochas- 
tic variable following a continuous-space continuous-time diffusion process. Let P(x, t) de- 
note the probability density of finding the system in state x at time t. In what follows 
we shall assume that the master equation (Chapman-Kolomogorov equation) governing the 
dynamics of P(x, t) can be represented by the following Fokker-Planck equation [3"3] : 

where 

J(x, t) = -£>(x) [eVP(x, t) + F(x)P(x, £)] (6) 

is the probability current. In equation (J6|, P(x) is the diffusion coefficient, F(x) is the force 
(not necessarily conservative) acting upon the system, and e is a parameter which will serve 
as our "temperature". For fluctuations of isothermal molecular systems in equilibrium at 
temperature T, Einstein's relation dictates that e = ksT, where ks is Boltzmann's constant. 
However, in the present work, the notion of temperature does not exist. 

We shall assume that the system can be driven and approaches to a nonequilibrium 
steady state in infinite time [21 E]. The nonequilibrium driving force comes from a "chemical 
driving force" in V x F(x) ^ [211 122] • When F(x) is conservative, 

V x F(x) = F(x) = -W(x). 

Then, the stationary P s (x) = e~ v ^^ e , while the stationary J(x) = 0. Furthermore, the 
stationary distribution P s (x) complies with detailed balance. That is, P s (x) is analogous 
to thermodynamic equilibrium [23] • Hence, a stationary system (|5p is also mathematically 
called equilibrium in this case [51 E] • 

Let us assume that Eq. ^ has one single ergodic stationary solution P s (x) with the 
corresponding stationary current V- J s (x) = 0; but usually, J s (x) ^ 0. We shall again write 
the stationary probability density as 

P*(x) = Cexp(-vI/ e (x)/e), (7) 



where function 1 i r e (x) is known as the non- equilibrium potential [23], C = 
[J Q dxexp (— \I/ e (x)/e)] 1 is a normalization constant, and Q represents the region of the 
state space available to the system. Note that in general ^/ e (x) is actually also a function 
of e. However, for many interesting applications, ^(x) is a function of x alone in the limit 
of e — > 0. The probability current J s (x) is a time-invariant, divergence-free vector field for 
the stationary distribution P s (x) [3]. 

B. Small e limit 

Let us consider first the case where F(x) is conservative. When e = 0, the system 
dynamic behavior is dictated, in a deterministic fashion, by the potential V^(x). That is, 
depending on the initial condition, the system state will evolve towards one of the local 
minima of V(x) and will remain there indefinitely. In that sense, every local minimum of 
V(x) corresponds to a stable steady state. Moreover, each stable steady state has a basin 
of attraction associated to it. Whenever the initial condition lies within a given basin of 
attraction, the system will eventually reach the state corresponding to the local minimum 
V(x) point. Finally, all neighboring basins of attraction are separated by saddle points and 
separatrices which the system has to surpass in order to go from one basin to the other. 

If e is not zero, but very small as compared to the height of the saddle points and 
separatrices between basins of attraction, the stationary probability distribution P s (x) will 
present high narrow peaks around the stationary states, and will attain very low values at 
the saddle nodes separating neighboring attractive basins. This further implies that the 
transition rates between every two attractive basins are small as well, as compared with the 
probability relaxation-rates within each basin. 

In the case of a non- conservative force, the stationary distribution depends on the non- 
equilibrium potential ^ r e (x), when it exists, in an analogous way as P s (x) depends on V(x) 
for conservative forces [25J. This means that the above considerations could be still valid 
when F(x) is non conservative. In particular, \P e (x) defines a landscape in the state space 
[S], a basin of attraction can be identified around each of the local minima of ^(x) and, in 
the small e limit, P s (x) presents high narrow peaks around each minimum of ^/ e ( x ) and takes 
very low values at the saddle points and separatrices that separate neighboring attractive 
basins. See [H] for systems with limit cycles. 
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IV. PROBABILITY DISCRETIZATION 



A. Discretization of the state space 

Consider a system whose non-equilibrium potential 1 J r e (x) has N local minima with the 
corresponding basins of attraction in the state space. Let Ui be the region of the state space 
delimited by the attractive basin of the ith local minimum of ^ e (x), and let Ej denote the 
boundary of Wj. In Appendix [B] we demonstrate that Sj can always be written as 

N 

Si = [J Eij, (8) 
3=0 

where S^- = (j = 1, 2 . . . N) represents the common boundary between Ui and Uj, while 
Sj denotes the part of the Ui boundary not shared with any other region. In case that 
and Uj share no boundary, S.y = 0. 

From the above considerations, the probability P, that the system state is in region Ui is 

Pi(t)= ! dxP(x,t). (9) 

Furthermore, it follows from ^ and Stokes' theorem that 

In the derivation of the above equation we have assumed that the probability current J is 
zero along S i0 . 

Let us analyze the integral L ds ■ J. From (6), it can be rewritten as 

f ds ■ J = - I ds- [eD(x)VP(x, t)} H(-ds • [eD(x)VF(x, t)}) 
ds • [eD(x)VP(x,f)] R(ds ■ [eD(x)VP(x,f)]) 
+ / ds ■ [D(x)F(x)P(x,t)] H(ds ■ [P(x)F(x)P(x, £)]) 
+ j ds- [P(x)F(x)P(x, *)] H(-ds ■ [P(x)F(x)P(x, *)]), 

JSij 

with H(-) being Heaviside's function. Given that H(x) > if and only if x > 0, just one of 
the first two terms in the right hand side of the previous equation is positive or zero, while 
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the other is negative or zero; the same is true for the last two terms. Let us define Jy as the 
sum of the two positive terms, and as minus the sum of the two negative terms. Hence, 

ds ■ J = Jij — Jji. (11) 



From its definition, > for all k, I = 1,2 ... N. Furthermore, from Eq. (11 ), Jf-i can be 



interpreted as the net probability flux from into U[. Finally, by substituting Eq. (11) 



into Eq. (10) we obtain 

^ = £(-W*)- (12) 

j'=i 

B. Adiabatic approximation and Kramers theory 



Following van Kampen [2n], Risken [27] and Freidlin-Wentzell [22] we make use of the 
assumption that e is much smaller than the height of the saddle nodes between every two 
attractive basins so that the corresponding transition rates are very small, as compared with 
the probability dynamics inside each basin. In consequence, the probability distribution 
within any can be approximated by the quasi-stationaty distribution 

P(x,t)^Q(t)exp(-^(x)/e), (13) 

with Ci(t) given by 

Ci ® = T a P f\,( wv (14) 

so that J w dxP(x, t) = Pi(t). From the approximation above, and a theorem from [25] that 
justifies the application of Kramers' theory to any pair of adjacent i and j [26J [27], it follows 
that 

Jy(t)= 7 ^(t), (15) 

where the transition rates 7^ are determined by the so-called local pseudo-potential; partic- 
ularly, by the height of the saddle points between neighbouring i and j attractors [HI EE] • 



Finally, by substituting Eq. (15) into Eq. (12) we obtain the following master equation 
for Pi(t): 

^f = E7^W-7^(i). (16) 
Note that, since in the stationary state Jfj = Pfjij, but J-j 7^ in general, Pfjij 7^ Pfjji- 



Therefore the emergent master equation in (16) is not necesarilly detail balanced 



V. THERMODYNAMIC STATE FUNCTIONALS 



A. Internal Energy 



Under the assumptions that the system modeled by Eq. ^ has a unique stationary 
distribution one can mathematically define, following Kubo [21] and many others including 
Ge and Qian pQ, the energy function associated to state x via the stationary distribution 
P s (x) as 

0(x) = -elnP s (x). (17) 
In systems with detailed balance, P s (x) equals the thermodynamic-equilibrium probability 



distribution P e (x) and Eq. (17) is equivalent to Boltzmann's law — provided we choose the 
zero level of free energy such that the partition function equals one. When detailed balance 
is not fulfilled, Kubo et. al. called </>(x) a stochastic potential [21] . Finally, from (17), the 
mean "energy" of the mesoscopic state P(x, f) can be written as 

U{t) = [ dxP(x,t)0(x) = -e f dxP(x,f)lnP s (x). 
Jn J n 



(18) 

Given the definition of the attractive basins, Q = [_} i=1 oui, while cui f)ojj = ® for all % ^ j. 



Then, Eq. ( 18 ) can be rewritten as 
17(f) = 



N N 

e^P,(f)lnP/-e^P,(f) / rfx 

1=1 i=l 



7 3 (x,f) 1 P a (x) 
Pi(t) 



Pf 



(19) 



with P/ = f u <ixP s (x). Finally, substitution of Eqs. (13) and (14) into Eq. (19) leads to 

N N 

U(t) = -e ^ Pi(t) lnP? + J2 Pi(t)Si, (20) 



i=i 



i=i 



where 



exp(-0(x/e)) exp(-0(x/e)) 
ax in . 



(21) 



and Z{ = f w dxexp(— 0(x/e)) . The first term in the right hand side of Eq. (20) can be 



interpreted as a coarse-grained contribution to the system's internal energy, arising from the 
distribution of probability among the ./V available attractive basins. On the other hand, the 



second term in the right hand side of Eq. (20) corresponds to the fine-grained contribution 



to the system internal energy, due to the distribution of probability density P(x, t) within 
each basin. The Boltzmann-like form of the terms within the integral originates from the 
adiabatic approximation we have made. 



B. Entropy and Free Energy 



The Gibbs entropy is denned as usual: 

S(t) = -e I dxP{x,t)\nP(x,t). (22) 
Jn 

By following an analogous procedure to the one in the previous subsection, known as the 



chain rules for entropy and relative entropy, Eq. (22) can be rewritten as 

" P(x,t), P(x,f) 



S(t) = -e^minPS-e^W) [ dx^^ln 

N N 



Pi{t) m) ' 
i=i i=i 

Once again, the entropy can be decomposed into a coarse-grained contribution — due to the 
distribution of probability among the iV available attractive basins — as well as a fine-grained 
contribution due to the distribution of the probability density P(x, t) within each basin. This 
result is in complete agreement with that in Eq. Q. Notice that, because of the adiabatic 
approximation, the fine-grained contributions to both U and S happen to be equal. 
From its definition, the mean Helmholtz free energy is 

F(t) = U(t) - S(t) = e [ dxP(x, t) In (24) 

Jn r (xj 

Furthermore, after performing the separation into coarse- and fine-grained contributions 
we obtain 

- p.m 

i=i « 

Observe that, in this case, the fine-grained contribution is absent, the reason being that the 
corresponding terms in U and S cancel at the time of subtracting. 

C. Further thermodynamic significance underlying the scale separation 

In the coarse-grained perspective one can define 

Ui = -elnP t s . (26) 



Hence, from Eq. (20) 



iV 



U(t) = y £ / P i (t)(u i + 5 i ). (27) 



8=1 
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Since U(t) is the average internal energy, we must have 



N 



(28) 



with Ui the mean internal energy associated to the basin Wj. Therefore, it follows from Eqs. 



(27) and (28) that 



Ui Ui Si . 



We see from this last expression, and the fact that is an entropic term (21), that Ui takes 



the form of a conditional free energy. Finally, we have from Q, (17), and (26) that 
in agreement with Eq. (J3]). 



VI. TIME EVOLUTION AND THERMODYNAMIC PROCESS VARIABLES 



A. General case 



By differentiating Eqs. (20), (23), and (25) and making use of Eq. (16) we obtain the 



following expressions for U, S, and F: 

N 



i,j=l ^ 1 



N 



N 



Pi 



ps p. ' 



(29) 
(30) 
(31) 



Before proceeding any further, notice that the formulas for U and S possess both coarse- 
and fine-grained terms. Nonetheless, because of the adiabatic approximation, the fine- 
grained terms in U and S are equal. Hence, they cancel in U — S and, in consequence, 
the time derivative for the free energy (F) is the same no matter wether the system has a 
fine-grained structure or not [TJ [13] . 

Following a procedure completely analogous to that in [dj we introduce the following 
definitions for the entropy production rate (e p ), the heat dissipation rate (Qd), and the 
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housekeeping heat (Qhk)' 

>j/'r,;-/^,)i» % J '- (32) 



TV 



Q d = \ E - ^) (in ^ + ^ ) , (33) 



N 



Q hk = lYl ( p rrn ~ Pan) in (34) 

lj=l 1 J 



It is straightforward to prove from the definitions above and Eqs. (29)-(31) that 



U — Qhk — Qd, S — e p — Q d , F — Q hk — e p . (35) 

As discussed elsewhere Pfl[2H], an d mentioned above, e p represents the entropy production 
rate of the system, the heat dissipation rate, and Qhk the energy influx rate necessary 
to keep the stationary distribution away from thermodynamic equilibrium. 



Given that the stationary distribution satisfies (Eq. 16) 

N 
3=1 

it is not hard to prove, see [20J for a detailed demonstration, that 

N p s N 

E ~ Pili ^ ln p =0 alld E ~ P ^ij)( S J ~ S i) = - 

i,j=l 1 i,j = l 

These results further mean that, in the steady state, 

N 

e P = Q d = Q h k= e -Yl (^ - PHj) In ^ > 0. (36) 
Z ij=i lij 

That is, all fluxes are larger than zero, but they balance in such a way that U, S, F = in 

the steady state. 

We point out that only Qd possesses a fine grained term. However, the correspond- 
ing discussion is delayed to the next subsection, in connection with the imposed adiabatic 
approximation and detailed balance. 

B. Detailed balance 

In the particular case where the stationary distribution P s (x) complies with detailed 
balance, the probability flux is null (3 s = 0) for every x [231 E6j |27]. This, together with 
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Eq. (11), further implies that Jfj = for all i ^ j. And so (Eq. 



15) that 



(37) 



Substitution of this last equation into Eqs. (32)-(34) gives 

N 



Y t {Psia-Piiis)^ P,P; 



«J=1 
N 



PiP* 



Qd 

Qhk 



Sj 



o. 



(38) 

(39) 
(40) 



It then follows from Eq. (35) that 



U = -Q d , S = e p -Q dl F = -e p . 



(41) 



By substituting Pj = P? into Eqs. (38)-(40) and taking into account Eq. (37), we have 
that 

= Qd = Qhk = 0. 

That is, when detailed balance is satisfied (or equivalently, when the system is in ther- 
modynamic equilibrium), all state variables remain constant in time because all fluxes are 
null. 



Consider again the adiabatic approximation. We can see from Eq. ( 13 ) that it is equiv 



alent to assuming that the probability distribution immediately evolves, within each Ui, to 
a local quasi-stationary distribution compatible with thermodynamic equilibrium. This last 



fact explains why neither e p nor Qkh — Eqs. (32) and (34) — possess fine-grained terms. 



C. Emergent coordinate and mean-field approximation 

The coordinate system of the phase space of a given stochastic dynamics, (x,y), usually 
is not the most natural one in terms of the multiscale dynamics. Fig. [T] illustrates how a 
dynamically natural coordinate system (r, s) can emerge from slow and fast manifolds. The 
slow manifold is widely known in chemical reaction dynamics as the "reaction coordinate" . 
The potential of mean force along the slow manifold, A(r), is widely called the "energy 
landscape" . 

13 



FIG. 1: Schematic representation of how a natural coordinate system (r,s) can emerge from the 
slow and fast manifolds of a given dynamical system originally described in the phase space (x, y). 



In terms of the emergent dynamic coordinates (r, s), the partition function is 



Z(e) = dx dy e - v{x ' y)/£ = / dr e~ Mr)/e , 



in which 



Mr) 



-eln / ds 



D(x,y) 



D(r, s) 



-V(r,s)/e 



(42) 



(43) 



with V(r, s) = V (x(r, s), y(r, s)). Furthermore (r, s) is a coordinate transformation of (x, y): 
x = x(r,s),y = y(r,s), with a non-singular Jacobian ||.D (a;, y)/-D(r, s)\\ ^ 0. 

How to discover the dynamically natural slow coordinate? One of the most widely used 
approaches is the mean field method. To illustrate this approach, consider the conditional 
free energy 

A x (x) = -eln J dy e - y ^ v)l \ (44) 
and also the conditional mean value for y, (y) x = E [y\x\: 



<y>. 



Jdyy 



-V(x,y)/e 



?A x {y)/e 



dyy e 



(45) 



/ dye~ v{ - x ^l e 

The curve (y) x can be considered as an emergent reaction coordinate. Then, using x as a 
parameter, A x (x) and (y) x give an "energy function" along the reaction coodinate. One can 
in fact choose a new coordinate r along the curve y = (y) x . 



It is easy to verify that (see Eq. 42): 



dx e~ Ax( - x)/e = dr e ~ Ar(r)/e = Z(e). 



(46) 
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All the equations so far are exact. However, in studies of real chemical and biophysical 



problems, one often chooses not to compute the last integral in (46). Rather, one finds the 



local or global minima of A r (r). The reasons for this practice are twofold: 

• First, it is often analytically impossible to carry out the integration. In this case, 
finding the global minimum (r*) is a reasonable approximation, especailly for small e 

-eln J dre- A ^ = A r (r*) - \ In (J^) +■■■. (47) 

Since the approximation neglects the fluctuations in r around r*, the method is widely 
called mean field approximation. In applied mathematics, this is known as Laplace's 
method for integrals [30] . 



Second, A r (r) might have multiple minima, say two. In that case, carrying out the 
integration is not as insightful as to identifying the bistability of the system, and the 
associated transitions. They can be visualized by the potential of mean force A r (r). 
In that slow, emergent stochastic dynamics on A r (r) arises. Both Flory- 

Huggins theory of polymer solutions [21] and the Bragg- Williams approximation for 
nonequilibrium steady-stat [12] are successful examples. 



VII. CONCLUDING REMARKS 



In this work we have studied the thermodynamic consistency, or invariance, across scales 
of a continuous-state continuous-time system, undergoing a Markovian stochastic process. In 
particular, we tackled the question of how the system thermodynamic variables, as well as the 
relations among them, transform when the system is described, in a coarse-grained fashion, 
by means of discrete variables. In that respect, we proved that, in the Helmholtz free-energy 
perspective, the thermodynamics derived from the continuous underlying detailed dynamics 
is exact. I.e. it is the same as if one only takes a middle-road and starts with a discrete 
description, with the transition rates 7^ either directly measured, or estimated by parameter 
fitting of experimental data. Below we further discuss some interesting consequences from 
these results. 
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A. Energy and thermodynamics across scales 



Consider a stochastic dynamical system with two levels of descriptions: an upper coarse- 
grained level and and a lower refined level with well-separated dynamic time scales. Then, 
following the analysis in the present paper, one has F\ = U\ — S\ and F 2 = U% — S2, 
where subscripts "1" and "2" denote upper and lower levels. Furthermore, U\ < Ui- Their 
difference is considered to be "heat" dissipated from the upper level to the lower level. 

The relationship between the classical Newtonian mechanics with S\ ~ and the molec- 
ular description of matter is an example. The energy difference U2 — U\ is entropic; it can 
not be fully used to "do work" at the upper level. The dynamics on the fast time scale 
at the lower level is considered to be "fluctuations" for the upper level. In a spontaneous 
transient at the upper level, diXJi — U\)jdt is the rate of the amount of energy being passed 
to the lower level. Energy conservation can only be understood from the description of the 
lowest level. Conversely, entropy is the concept required to characterize the changing U 
across scales. 

The thermodynamics across scales in stochastic dynamics, in particular the energy dissi- 
pation from a upper scale into a lower scale, has been a central unresolved issue in the theory 
of turbulence [32] . Whether the newly developed thermodynamic framework of stochastic 
dynamics can shed some light on the problem remains to be seen. 

B. Coarse-graining as conditional probability 

In a recent study [20] we have shown that the conditional free energy, which corresponds 
to the potential of mean force in continuous stochastic systems, plays an essential role in the 
invariance of mathematical irreversible thermodynamics of multiscale stochastic systems. 
Furthermore, in [33], we have proved that Legendre transforms between different thermo- 
dynamic potentials for different Gibbs ensembles can be derived in terms of conditional 
probability for a pair of random variables. As a matter of fact, one can consider coarse- 
graining as a special form of conditional probability. Treating {(x, z)|x 6 K M , 1 < i < N} 
as a pair of random variables, the coase-graining means 




ft(*)/Pt xew,, 
x ^ ue, 



(48) 
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in which fe(x) is the joint probability, /(x|£) is the conditional probability, and 

P t = f dxf t (x) = [ dxf t (x). (49) 



Then, the standard chain rule for free energy (i.e. relative entropy) |18j . 
at f ( ) N p N / r 



my 



(50) 



takes an interesting, equivalent form: 

N „ AT 



in which 

is the "conditional free energy" of the sub-system £. For subsystems with rapid steady 



state, this term is zero. Thus, a system's total free energy (50) is the free energy of the 
coase-grained system. 
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Appendix A: The entropy depends on how finely the system is described 

Consider the definitions for S(e) and S(e) given in Eq. and rewrite them as 

S(e) = — eNpjlnp;, and S(e) = — e / <ixp(x) lnp(x), 

Jn 

with 

e -Ai/e e -V(x)/e 

Pi = ~^7V> and P( x ) 



Z{e) ^ ' Z (e) 

It follows from Eqs. 0-([3]) and the former definitions that 

Pi = / rfxp(x), 

JuJi 

in which U i=1 = Q. We can now use this last result to rewrite S(e) as 

>(x) 



S(e) = S{e)-e^p i J u dK^j In (i 
However, since J ctxp(x)/pi = 1, 



Pi 



SM * -e I dx l'^ In ^ I > 



Pi J \ Pi 

is a conditional entropy associated to the probability distribution within Wj. Hence, 



5(e)- 5(e) = 0- 



Appendix B: Analysis of the boundaries of the regions Ui covering £1 

We start with some definitions. Let X be an arbitrary set in W 1 . The closure of X, 
Cl(X), is defined as the intersection of all closed sets C such that X C C. On the other 
hand, the interior of X, In(X), is defined as the union of all open sets A such that A G X. 
Finally, the boundary of X, Bd(x), is defined as Bd(X) = Cl(X) \ In(X). 

Let Q G M n be an open set and {ui} a cover of Q such that u; C O for all i, and 
Q = [jf =1 uJi. We make the following assertions regarding Q and {cuj}: 
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1. Observe that, since the union of closed sets is closed, Q C \jf =1 Cl(u>i), with 
[jf =1 Cl(uji) a closed set. Furthermore, Cl(Q,) C [jf =1 Cl(u>i) 



2. Note also that Uj D 



3. Moreover, given that is open, u>i D D, with D = Q \ {jj^Cliuj) an open set. And 
so, D C In(uii). 

4. From the definitions above, the boundary of Ui, Sj = Bd(ui), is a closed set contained 
in C7(f2). Hence, from Assertion 3, In(u>i) = D [jln(u)i) and, from the definition of a 
set boundary, 

Sj = C/(wj) \ Info), 

= Cl(ui)\ [D\Jln(co t )] , 

= [s.fic/^)] \l>, 

= ^i\D. 



5. Let us define now = [Jj^ Cl(ujj), so D = Q\H. From this: 

Hi = ^ \ D, 



(Bl) 



(B2) 



6. We have from the definition of H that 



s i = [H i \n]U 



Recall that Q is open, so Sj \ Q is the part of the closed set lying outside Q. On 
the other hand, Sj P| Cliuij) is the part of Sj lying within the closure of Wj. 
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7. Notice that 



3)\i 



= cz^ofifsAM^)]- 



8. Furthermore, 

Sj \ In(u)j) = [Cl(ui) \ In(u)i)\ \ In(uj), 
= [Cl(u t )\In(u,)]\In(uj^. 

9. If we assume now that Cl(u>i) and In(uj) are disjoint, then Cl{uji) \ In(uj) = Cl(ui). 
Then, from Assertion 8, 

Ei \ In(ujj) = Cl{uji) \ In(uji) = Sj. 
Moreover, from Assertion 7, Sj \ In(u}j) = Cl(u>i) P|Sj 

In conclusion, if we assume that G M n is an open set such that Q = IJj 60 ^ and we 
assume as well that Cl(ui) and In(uj) are disjoint for all i ^ j then, from Assertion 6, 



s i = [H i \n]U 

Denote E i0 = 'E i \Q and S^- = Sj P| Sj, so 



i=o 



□ 
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